SHEP-09-08 



A new family symmetry for SO (10) GUTs 

Stephen F. King* and Christoph Luhn^ 

School of Physics and Astronomy, University of Southampton, 
Southampton, SO 17 IB J, United Kingdom 

Abstract 

We argue that the projective special linear group PSL2{7), also known as E(168), 
has unique features which make it the most suitable discrete family symmetry for 
describing quark and lepton masses and mixing in the framework of 50(10) type 
unified models. In such models flavon fields in the sextet representation of PSL2{7) 
play a crucial role both in obtaining tri-bimaximal neutrino mixing as well as in 
generating the third family charged fermion Yukawa couplings. In preparation for 
physical applications, we derive the triplet representation of PSL2{7) in the ba- 
sis S, T, U, V where S, T, U are the familiar triplet generators of 5*4 in the diago- 
nal charged lepton basis where T is diagonal. We also derive an analogous basis 
for the real sextet representation and identify the vacuum alignments which lead 
to tri-bimaximal neutrino mixing and large third family charged fermion Yukawa 
couplings. 
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1 Introduction 



It has been one of the long standing goals of theories of particle physics beyond the 
Standard Model (SM) to predict quark and lepton masses and mixing. With the discovery 
of neutrino masses and mixing, this quest has received a massive impetus. Indeed, perhaps 
the greatest advance in particle physics over the past decade has been the measurement of 
neutrino masses and mixing involving two large mixings associated with atmospheric and 
solar neutrino oscillation experiments, while the remaining mixing, although unmeasured, 
is constrained by reactor neutrino oscillation experiments to be relatively small. The 
largeness of the two large lepton mixings contrasts sharply with the smallness of quark 
mixing, and this observation, together with the smallness of neutrino masses, provides 
new and tantalising clues in the search for the origin of quark and lepton flavour. 

It turns out that the observed neutrino oscillation parameters are consistent with a 
tri-bimaximal (TB) mixing structure [1]: 



It has been observed that this simple form might be a hint of an underlying family symme- 
try Gf, which preserves certain group transformations (corresponding to the generators 
5", [/, as discussed later) which are respected by tri-bimaximal neutrino mixing, and sev- 
eral models have been constructed that account for this structure of leptonic mixing [2-6]. 
It is possible to extend the underlying family symmetry to provide a description of the 
complete fermionic structure [7-2 ij^, in which, in contrast to the neutrinos, the quarks 
have a strongly hierarchical structure with small mixing. 

A desirable feature of a complete model of quark and lepton masses and mixing is 
that it should be consistent with an underlying Grand Unified Theory (GUT) structure, 
either at the field theory level or at the level of the superstring. The most ambitious 
Gf ® GUT models which have been built to achieve this are based on an underlying 
5*0(10) type GUT structure. This is very constraining because it requires that all the 16 
spinor components of a single family should have the same family charge, comprising the 
left-handed fermions and the CP conjugates of the right-handed fermions ip'^, including 
the right-handed neutrino. We emphasise that it is very desirable that the right-handed 
neutrinos should be unified with the rest of the quarks and leptons, since this feature 
leads unavoidably to neutrino masses. By contrast, in SU{b) right-handed neutrinos are 
gauge singlets without which it is natural to have zero neutrino masses. Although it 
seems somewhat ad hoc to add right-handed neutrino singlets to SU{b), once they are 
present it is straightforward to construct models of lepton masses and TB mixing that are 
consistent with quark masses and mixing, and there are several successful models of this 
kind [11]. Nevertheless, it is worth remembering that in such models the right-handed 

^ See [22] for review papers with more extensive references on neutrino models 
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neutrinos must be added by hand as an optional extra and therefore neutrino mass is 
not strictly a prediction of SU{5) whereas in SO (10) neutrino mass is unavoidable since 
the right-handed neutrinos are unified with the rest of the quarks and leptons, leading 
to the prediction of three right-handed neutrinos. In fact similar comments apply to any 
partially unified model in which SU{2)r is gauged, for example Pati-Salam; also a gauged 
U{1)b-l predicts right-handed neutrinos due to anomaly considerations. 

Despite the theoretical attractiveness of a G/ ® GUT structure of the form G/ (g> 
5*0(10), there are very few models in this class that are capable of predicting, or more 
accurately postdicting, TB mixing. As we shall discuss later, in order to obtain TB mixing 
naturally (i.e. enforced by a symmetry) one requires a group with a non-Abelian structure 
containing triplet representations, together with a particular set of discrete group elements 
which embody the symmetry properties of the TB mixing matrix in Eq. (11.11) . There is also 
an additional theoretical requirement in 5*0(10) that the family symmetry must contain 
complex triplet representations as discussed below. Examples of such models based on 
the Pati-Salam subgroup of 50(10) have been constructed where the family group Gf is 
SU{3)f [21] or A(27) [19]. In such models all 16 components of a single family including 
the left-handed fermions and the CP conjugates of the right-handed fermions ip'^ are 
assigned to the complex triplet representation of SU{3) / or A(27), namely ipi ~ 3, ~ 3. 
Flavon anti-triplets 0* ~ 3 are introduced to break Gf, and the lowest dimension Yukawa 
operators consistent with an SU{3) f family symmetry are then. 



where H is the Higgs field which is generally taken to be a singlet if ~ 1 under Gf. For 
discrete family symmetries like A(27) additional contractions may in principle become 
possible, however, in concrete models they are usually assumed to be suppressed or absent. 
If a particular flavon 03 has a vacuum expectation value (VEV) aligned along the third 
direction (^s) = {0,0,V), then this operator would imply a Yukawa matrix which is 
suitable for describing the large third family Yukawa coupling. 



On the other hand, if the triplet representations were taken to be real rather than 
complex then this would allow an undesirable alternative 50(3) invariant contraction of 
the indices in the Yukawa operator in Eq. (II. 2p . arising from the fact that for real triplets 
3x3 contains a singlet 1 (whereas for complex representations it does not), namely 
ilji'ilj^(f)^ (f}' H leading to a Yukawa matrix proportional to the unit matrix which would tend 
to destroy any hierarchies in the Yukawa matrix. For real representations, there is no 
symmetry at the effective operator level that could forbid such a trivial contraction of the 
two triplet fermion fields, together with the trivial contraction of the two triplet flavon 
fields. However in principle it is possible to appeal to the details of the underlying theory 
responsible for generating this operator in order to forbid the trivial contraction, which 
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would involve a discussion of the heavy messenger states whose exchange generates the 
operator, though we shall not pursue this possibility further here. 

Alternatively, there is the possibility of obtaining a Yukawa operator from a single 
triplet flavon field, (j)'^, using Sijkip^ipjCj)).!!, where Sijk is some tensor contraction. For 
example if the fields are triplets of SU{3) or 5*0(3), then we can have Sijk = eijk, the totally 
antisymmetric tensor. However such operators cannot generate the diagonal elements of 
the Yukawa matrix. For discrete groups such as A4 the tensor Sijk can additionally be 
chosen totally symmetric. Depending on the basis of the triplet representation, it may 
however still be impossible to generate the diagonal elements of the Yukawa matrix (see 
e.g. [13]). Therefore, in typical models of this kind based on A4 5*0 (10), the singlet 
contraction ipiip'^'qH , where t] is an ^4 singlet, are typically invoked to obtain the diagonal 
elements, and the problem of achieving a hierarchical Yukawa matrix is challenging [12]. 

The above considerations seem to point towards a, G j ® S0{1Q) theory, where Gf 
is S'f/(3)/ [21] or A(27) [19] since these groups contain complex triplet representations. 
However there is one undesirable feature of such theories, namely that the top Yukawa 
coupling must necessarily originate from a higher order operator involving two flavon 
fields, as in Eqs. f 1 1.2 1) and (11.31) . This implies that the flavon VEV must be quite close in 
magnitude to the mass scale of the operator leading to a convergence problem, since the 
effective top quark Yukawa coupling ht is typically about 0.5 at the GUT scale in SUSY 
models, which implies that ht = yV'^/M'^ 0.5, or V/M ^ 0.7?/~^/^. We emphasise 
that this problem is unavoidable in Gf ^ S'0( 10) models unless the Higgs fields carry Gf 
quantum numbers which would allow renormalisable Yukawa couplings, but such theories 
face the challenge of flavour changing neutral currents. 

There is a further problem arising from anti-triplet fiavons in the explicit SU{3)f [21] 
or A (27) [19] models. It is related to the origin of the top and bottom quark Yukawa 
coupling due to the diagram in Fig. [T]^a) which is mediated by SU(2)l singlet messengers 
S*^ which are supposed to be lighter than SU{2) l doublet messengers and hence give the 
dominant contribution to the effective Yukawa operator. Fig. [I](a) involves anti-triplet 
fiavons and is mediated by heavy fermionic messengers of two types namely 
and with mass Mu and Md, corresponding to the diagrams involving up and down 
quark CP conjugated right-handed fields If^ and D'^. Similar diagrams for the first and 
second family quarks require a messenger hierarchy in the up and down Yukawa sectors, 
Mu/M£) 3, which is necessary to account for the two different expansion parameters in 
the two sectors, e„ = v/Mjj ^ 0.05 and = v/Md ^ 0.15, where v refers to the first and 
second fiavons. This must be corrected for in the third family to achieve ht ^ hi, ^ 0.5. 
This is done in a rather cumbersome way by assigning the third family flavon ^3 to be a 
combination of a singlet plus a triplet of SU{2)ji, and adjusting the flavon VEVs to lead 
to ht ^ h}y K, 0.5 [19,21], where the larger values of these Yukawa couplings is achieved 
by assuming larger third family flavon VEVs V > a.s discussed previously. 

It is possible to alleviate both the above problems by invoking (anti-)sextet flavon 
fields as the origin of the third family Yukawa couplings. For example, introducing a 
two index anti-sextet of SU{3)f, x*-', which is symmetric under interchange of i ^ j, the 
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Figure 1: Diagrams responsible for the Yukawa couplings. With anti-triplet flavons, 
case (a), the messengers are SU{2)l singlets; with (anti-)sextet flavons, case (6), the 
messengers are SU{2)l doublets. 



lowest order Yukawa operators become, 

j^r^H ■ (1-4) 

If the anti-sextet fiavon x has a VEV aligned along the 33 direction (x*"*) = V5i^5j^, then 
this operator would imply a Yukawa matrix of the form 



F = , (1.5) 




with the top quark Yukawa coupling of 0.5 implying V/M 0.5?/"^, which has somewhat 
improved convergence properties. In addition, unlike in [19,21], it is sufficient to assume 
that the third family fiavon x is a singlet of SU{2)r since the messengers will be qualita- 
tively different in the case of the third family due to the fact that x is a fiavon anti-sextet 
rather than an anti-triplet, as we now discuss. 

According to the above discussion it is plausible to assume that the operators respon- 
sible for the third family Yukawa couplings involve (anti-) sextet flavons and are generated 
from diagrams involving messengers S which carry Higgs quantum numbers as well as be- 
ing (anti-) sextets, see Fig. WLh)- Such Higgs messengers would imply similar mass scales 
for the top and bottom Yukawa operators, naturally and elegantly leading to top-bottom 
unification, with the (anti-)sextet fiavons being 5*0(10) singlets. The operators for the 
first and second family quarks could then continue to involve anti-triplet fiavons, with 
the relevant operators, given in [21], arising from diagrams involving heavy fermionic 
messengers of mass Mu and M/j, similar to Fig. [I](a). 

The above discussion provides a good motivation for considering fiavon (anti-) sextets 
as the origin of the third family Yukawa couplings in Gj ® 5*0(10) models. Although 
this is possible in the case where the family group Gf is 5*^7(3)/ [21], it is not possible 
for the A(27) [19] models for the simple reason that A(27) does not admit (anti-)sextet 
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representations. On the other hand, discrete family symmetry groups have been shown to 
be preferred to continuous family symmetry groups such as SU{3)f. The reason is that, 
starting from a discrete symmetry group which contains a finite number of group elements, 
it is easier to achieve vacuum alignments which preserve the subgroup elements S, U 
required for tri-bimaximal neutrino mixing, rather than starting from a continuous group 
which contains an infinite number of group elements. The question then naturally arises 
of whether there is a discrete subgroup of SU{3)f which admits triplet representations, 
to which we can assign the three families of quarks and leptons, together with (anti- 
) sextet representations suitable for accommodating the flavons responsible for the third 
family Yukawa couplings. We further require the triplet representations to be complex, 
to forbid the singlet contraction ipiip'^'qH, but the (anti-) sextet representation is allowed 
to be real or complex. The smallest discrete group consistent with these requirements is 
PSL2{7), which is the projective special linear group of two dimensional matrices over the 
finite Galois field of seven elements. PSL2{7) contains 168 elements and is also known as 
S(168) [23]. For practical purposes, the most important features of PSL2{7) are that it 
admits complex triplet and anti-triplet representations as well as a (real) sextet. PSL2{7) 
is a subgroup of SU{3), and contains 6*4 as a maximal subgroup. By contrast, A(27) is 
also a subgroup of SU{3) but is not a subgroup of PSL2{7), and does not contain 6*4 
as a subgroup. The relationship of PSL2{7) to some other family symmetries that have 
been used in the literature is discussed in [17, 24, 25] and depicted in Fig. [2l From a 
mathematical point of view PSL2{7) is the unique finite group that is both simple and 
contains complex triplet representationsJl 

In this paper we shall develop the representation theory of PSL2{7) = S(168) for 
triplets and sextets in a convenient basis suitable for applications of PSL2{7) as a family 
symmetry capable of describing quark and lepton masses and mixing in the framework of 
5*0(10) type unified models. The representation theory of PSL2{7) has been developed 
in [24] in terms of a standard presentation involving two generators A, B. However, for 
physical applications, it is desirable to be able to relate the representations of PSL2{7) 
to those of 5*4 in a basis in which the generators of 5*4 are denoted as 5, T, U where 
S, U are the symmetry transformations of the tri-bimaximal neutrino mass matrix in 
the diagonal charged lepton mass basis. We shall show how the triplet representation 
given in terms of the standard generators A,B in [24] may be related to four PSL2{7) 
generators which we shall denote as S, T, U, V where 5", T, U are the triplet generators 
of its maximal subgroup 5*4 in the physical basis described above. In such a basis the 
subgroup structure PSL2{7) D 5*4 D ^4 just corresponds to the respective generators 
being 5, T,U,V D S,T,U D S, T. We shall also construct explicitly the four generators 
of PSL2(7) in a real sextet representation S,T,U,V which we relate to the couplings 
of the neutrino operators and third family charged fermions. We identify those sextet 
vacua which preserve the generators S, U as required for tri-bimaximal neutrino mixing, 
or alternatively break all the generators and give rise to a large third family charged 

^ There exists another simple group with triplet representations, the icosahedral group A^. Being a 
subgroup of S'0(3), its two triplets are real which gives rise to the trivial bilinear invariant. Moreover, 
^4 is not a subgroup of A5 so that tri-bimaximal mixing cannot be obtained naturally; instead the solar 
mixing may be related to the golden ratio [26] . 
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Figure 2: PSL2{7) = S(168) and its relation to some other subgroups of SU{3). A line 
connecting two groups indicates that the smaller is a subgroup of the bigger one. 

fermion Yukawa coupling, such as the top quark Yukawa coupling. 

The layout of the remainder of the paper is as follows. In section [2] we discuss the 
symmetries of the TB neutrino mass matrix in the diagonal charged lepton basis and shall 
discover from this bottom-up procedure the matrix transformations S, U which leave the 
TB neutrino mass matrix invariant and which correspond to two of the generators of 
5*4 in this physical basis. In section [3] we shall transform the triplet representations 
of PSL2{7) in the A,B presentation to the desired S,T,U,V form. In section H] we 
shall repeat an analogous exercise for the sextet, by defining a convenient real sextet 
basis S,T,U,V in terms of which the couplings involving a sextet flavon field may be 
simply expressed. Section [5] outlines the central steps in building a realistic PSL2{7) 
flavour model, whose construction, including a detailed discussion of the dynamics, will 
be presented in a separate publication [27]. We conclude our presentation of a PSL2{7) 
family symmetry in the framework of 5*0(10) type GUT models in section [61 



2 The symmetries of tri-bimaximal neutrino mixing 

The typical Lagrangian (or superpotential) of interest generically consists of two parts, 
the Yukawa sector and the Majorana sector. The effective Yukawa sector is of the form. 



while the effective Majorana sector is of the form 



(2.1) 



(2.2) 
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where YX^'' and F^f are Yukawa and Majorana couplings, respectively, while H are 
Higgs fields. When the Higgs develop their VEVs, and are identified with left-handed 
lepton fields L, while ip'^ are identified with charged conjugated right-handed charged 
leptons such as e^, the effective Yukawa operators lead to the charged lepton mass matrix 
Mfj oc Y^j, while the effective Majorana operators lead to a neutrino Majorana mass 
matrix My. oc K"^. 

In the fiavour basis, in which the charged lepton mass matrix is diagonal and the TB 
mixing arises from the neutrino sector, the effective neutrino mass matrix, denoted by 
My^, may be diagonalised as, 

^diag = U^B M^B Utb = Diag (mi, ms, ma) . (2.3) 
Given Utb-, this enables to be determined in terms of the three neutrino masses, 

M^^ = ^A+^5 + ^C, (2.4) 

where the three matrices are 








1 










1 










1 








A = \^-2 1 Ij, C=|0 1 -1|. (2.5) 

Prom above we may write M!^^ as the symmetric matrix, 

(a b c \ 

h d e \ , (2.6) 

c e /; 

where, 

2mi m2 nil 1712 , . . , rui 1712 

a — , b — c— , a—t — a + b — e — . (2.7 

33' 33' ■' 632^^ 

In particular b = c and d = f and a + b = d + e are the characteristic signatures of the 
TB neutrino mass matrix in the fiavour basis. 

It seems paradoxical that, in this basis, ^^"'^ seems to obey a completely different sym- 
metry as compared to that of While jC^'"'^ only respects a trivial phase symmetry 
corresponding to a diagonal phase transformation on the fields, which leaves the diagonal 
mass matrix invariant, respects a more subtle discrete symmetry which leaves 
M^^ invariant, even through the same fields, namely the lepton doublets L, are common 
to both. How can this be reconciled? If the underlying couplings F^J"^ and Y^"'-' are 
simply numbers, then there is no way that different parts of the same Lagrangian involv- 
ing the same fields could have different symmetries: it would not make any sense since 
any symmetry transformation on the lepton doublets L NL must transform both 
£Yuk £Maj ^ ^YYic resolution to this problem is intrinsically related to the origin of the 
Yukawa couplings. If the Yukawa couplings are generated by the VEVs of flavon fields, as 
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is generically the case for models which respect a family symmetry, then it is possible to 
have different symmetries in the Yukawa and Majorana sectors, providing different flavons 
appear, or are dominant, in the two sectors. The idea is that the complete high energy 
theory Lagrangian, including both and C^'^"-^ ^ would both respect some family sym- 
metry Gf due to the presence of flavons of two types (p^^^ and (f)^"-^ , where 0^"*^ appears 
in but not in where only 0^^'^-? appears. In other words the effective Yukawa 

sector is supposed to originate from terms of the form, 

Z:^"' = ^^(f(r^r,H , (2.8) 
while the effective Majorana sector comes from terms like 

c''"' = ^.C^.m ' (2-9) 



where both terms are invariant under Gf, with the indices z,j now being regarded as 
indices of Gf. Generically 0^"^ and 0*^"-' may represent either a single flavon or a polyno- 
mial of flavons of a particular type. When the flavons develop VEVs the family symmetry 
is spontaneously broken, so that the full family symmetry is not apparent in either of 
the low energy Lagrangians C^'^^ or C^"-^ , However, although both flavon types break 
the family symmetry, if 0^"'^ and 0*^°-^ preserve two different subgroups of Gf, say G^"*^ 
and G^^""^ , then this will result in the two low energy Lagrangian terms C^'''' and C^^^ 
respecting different symmetries G^"^ and G^^""^ , as observed. 

In the case of TB mixing, we have seen that, in the diagonal charged lepton basis, 
after the Higgs develop their VEVs, the Majorana Lagrangian is 

C^S' = ^,{M^^),,^|J^ , (2.10) 

where M^^ is given in Eq. (12.41) . According to the discussion of the preceding paragraph, 
{M^^)ij oc {(pfj"'''), so it makes sense to identify the symmetry G^^-' respected by jC^^"' in 
Eq. (I2.10p . independently of the symmetry of the charged lepton sector. We want to find 
the most general symmetry transformation N, corresponding to ijj —>■ Ntp, which leaves 
C^^^-' in Eq. (I2.10p invariant. This will subsequently be identified with the symmetry 
respected by {4>ij"'''), which will be a subgroup of the full family symmetry Gf. Assuming 
that is a unitary matrix, the symmetry condition is given by, 

M^bN = NM^b, (2.11) 

with A^ denoting the complex conjugate of A^. Since the symmetry must be independent 
of the neutrino mass eigenvalues, Eq. (12. lip imphes, from Eqs. (12. 4p and (12. 5p . three 
independent conditions, 

AN = NA, BN = NB, C N = NG . (2.12) 

Starting with the most general complex (3 x 3) matrix, these conditions readily lead to 
the real matrix 

(rii n2 712 

n2 ni + n2-n^\ . (2.13) 

n2 ni + n2- 
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Since appears quadratically in the conditions of Eq. fl2.12p . it is only determined up to 
an overall sign. Assuming that has determinant +1, for definiteness, the requirement 
of unitarity (orthogonality) finally yields the most general symmetry of the TB neutrino 
mass matrix M^^ as the group obtained from the generators S, U, given by, 

-122\ /100\ 
2 -1 2 , t/=- 001. (2.14) 
22-1/ \010/ 

It is easy to show that this group has only four elements; excluding the identity, all are 
of order two. Hence we have discovered Klein's four-group Z2 x Z2 as the symmetry of 
the TB neutrino mass matrix. 

By contrast, in the diagonal charged lepton mass basis, C^^^ satisfies a rather trivial 
diagonal phase sjTiimetry. The first possible choice of phases which reversely enforce a 
diagonal charged lepton mass matrix corresponds to powers of the diagonal symmetry 
generator T, given by 

/I 0\ 

T= , (2.15) 

\0 LU J 

where 00 = e^'^*/^. 

As we will prove in the following section, the matrices S, T, U form the generators of 
the group 5*4 in the triplet representation, and thus, according to the previous discussion, 
5*4 or any group containing 5*4 becomes an attractive candidate family symmetry Gj. 

3 The PSL2{7) triplet in the physical S4 basis 

As outlined in the previous section, a natural family symmetry should contain the non- 
Abelian finite group 5*4, the permutation group on four letters. As both triplet repre- 
sentations of 5*4 itself are real one can form the trivial invariant from 3x3 and 3' x 3'. 
Therefore the hierarchy of the fermion masses cannot be explained (without fine-tuning) 
if we want to work in a unified framework where the quarks and leptons all transform as 
triplets 3 or alternatively 3' under the family group 5*4. 

Requiring complex triplet and (real or complex) sextet representations, the smallest 
finite groups that have S4 as a subgroup are A(96) and PSL2{7) = S(168). The former 
belongs to the series of groups A(6?t,^) and has one sextet, six triplet, one doublet and two 
singlet representations. It is interesting to note that two of the triplets are real while the 
other four are complex. The irreducible representations of PSL2{7) on the other hand 
are 

PSL2i7) : 1, 3, 3, 6, 7, 8 . 

As there are only two complex triplet representations (3,3), the group PSL2{7) appears 
more attractive from the model building point of view. Furthermore, as already mentioned 
in the introduction, it is the unique simple finite group with complex triplet representa- 
tions which makes PSL2{7) appealing also from the mathematical perspective. 
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In the following we want to pave the way for building flavour models based on the 
group PSL2{7). The main idea is to use the crucial ingredients of the known and 
models and generalise these in the context of a PSL2{7) symmetric theory. In particular, 
we have to find a formulation in which the groups A4, 6*4, and PSL2{7) can be easily 
compared. This involves two separate questions: 

1. What is the structure of the embedding A4 G S4 G PSL2{7) in terms of generators? 

2. What is the most convenient basis for the matrix representations of these generators? 



3.1 A4 C S4 

Before discussing the group PSL2{7), let us illustrate the situation for the less complicated 
case where A4 is embedded into 5*4. These groups belong to the series of groups A^Sn^) 
and A(6n^), respectively. With n = 2, we have 

A = A(12) = (^x^)^^, (3.1) 

c d a 

S4 = A(24) = (^x^)x(^x^), (3.2) 

c d a b 

where the generators c, d, a, and b of the various cyclic groups have to satisfy the following 
relations 



c 



d"^ = = 1 , cd = dc, aca? = cd , ada"^ = c , (3-3) 
52 = (a5)2 = 1 ^ bcb = d . (3.4) 



This presentation has the advantage that the embedding of A4 into S4 is explicitly given: 
adding a generator b which satisfies the relations of Eq. ( 13. 4p to the group A^ leads 
to 5*4. However, in practice, model builders adopt a slightly different convention for the 
presentation of A4. Denoting the generators by s and t, this alternative presentation takes 
the compact form 

A4 : <s,t|s2 = t^= (st)^ = l> . (3.5) 

The first observation is that it is sufficient to work with only two instead of three genera- 
tors. This can be easily understood by noting that the generator c is nothing but ada^, as 
required by the last relation in Eq. (13.31) . Therefore c can be replace by ada^ and becomes 
redundant in the presentation. Having dropped the generator c, we need to identify s and 
t among the 12 group elements of A4 given in terms of the original generators d and a. 
Since s and d are both elements of order two, while t and a are elements of order three, 
it is suggestive to set 

s = d , t = a . (3.6) 

One can show that this identification is consistent by inserting it into Eq. (13. 3p and 
rewriting the result to obtain the presentation of Eq. (13.51) . However, other identifications 
are equally possible. For instance, we could have chosen s' = c = ada? and t' = a which 
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is related to Eq. (13.61) by the similarity transformation s' = asa? and t' = ata?. In the 
following, we disregard such an ambiguity and stick to the identification given in Eq. (13.61) . 

When applying a non-Abelian discrete family symmetry, one is mainly interested in 
the matrix representajtions for the triplets. In the presentation of S4, with c, a, and h 
they are given in 




^[3] = 0-1 , a^^^ = 1 , (3.7) 





= ±010. (3. 



Notice that fel^l has two different matrix representations corresponding to the two triplets 
of 5*4. Of course, these explicit matrices are defined in a particular basis for the triplets. 
Even though ^4 models were originally formulated using the basis of Eq. (13.71) . a dif- 
ferent basis has proved more useful in physical applications. Suppressing the redundant 
generator c^^\ we define for the triplet representations 

S = w(P^w\ T = wa^^'^w\ U' = wh^^^w\ (3.9) 

with the unitary matrix w given by 




We then obtain the explicit 5*4 generators in the new basis 

. f-1 2 2 \ /I 0\ 

S = - \ 2 -1 2 , T= 0cj2 , f/'=±|0 0cj2|. (3.11) 
























^0 





UJ 




Now S and T are in the conventional form suitable for A4 model building. 5*4 is obtained 
by simply adding the generator U' or, alternatively, a product of U' with any group 
element of A4, e.g. 

/I 0\ 

U = T^U' = ±\0 1 . (3.12) 
\0 1 oj 

With the matrices S, T, and U we have recovered the generators of 5*4 for the triplet 
representations found by Lam in [29], with the identifications S = G2, T = (-Fi)^, and 
U = G3. Note that they are identical to the matrices in Eqs. (12.141) and (I2.15p . 



■^A different presentation for S4 is given in [16] hy < A, B \ — B"^ = (AB^)'^ = 1 >. Using the 
explicit matrix representation for the triplets, the generators A and B are related to a,b,c,d through 
A — bda and B — a? . It is straightforward to show that this presentations is equivalent to the one shown 
in Eqs. ((H^)) and 
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3.2 ^4 C PSL2{7) 

Having identified the embedding of A4 in 5*4 in terms of tfie pfiysically interesting genera- 
tors S, T, and U, we want to proceed with the discussion of the group PSL2{7). Details of 
its mathematical structure - e.g. character table, irreducible representations, Kronecker 
products - can be found in [24]. There, the presentation of PSL2{7) is given in the 
compact form 

PSL2{7) : <A,B\A'^ = B^ = {ABf = (A-^B'^AB)^ = 1 > . (3.13) 

In order to extract a set of 24 elements that constitutes 5*4 within the 168 elements of 
PSL2{7), we first identify the order three generators a and B. This choice can be made 
without loss of generality since all order three elements belong to one conjugacy class and 
are therefore related to each other by a similarity transformation. With this choice, the 
5*4 generators d, a, b are related to the PSL2{7) generators A, B by 

d = [{AByB{AB)]\ a = B, b = B\AB){ABY[{AB)^B{AB)]\ (3.14) 

Relying only on the PSL2{7) presentation of Eq. (13.131) . these identifications can be 
shown to satisfy the relations in Eqs. (13. 3p and (13.41) regardless of a concrete matrix 
representation. The complete group PSL2{7) can be obtained by adding a generator v 
which, for later convenience, we choose to be 

V = ibdab) Aibdab)'^ . (3.15) 

It is worth noting that, contrary to A4^ and 6*4 which are given in Eqs. (13.11) and (13.21) . 
PSL2{7) cannot be written as a semidirect product of Z„ symmetries because it is a simple 
group. Nonetheless, PSL2{7) can be obtained from one order three and three order two 
generators: 

= 1 , d^ = b"^ = = 1 . (3.16) 

Here, we refrain from presenting the other conditions that these four generators must 
satisfy since they are rather involved and do not provide any new insight into the group 
structure. 

As mentioned above, we are mostly interested in triplet representations. The (3 x 3) 
matrices A^^^ and B^^^ were derived in [24]. Defining 77 = e^'^*/^, they take the form 

rf ~ 7] 7] — 1 

51^1 = -^\r]^-l r]^-T]^ r]^-r]^\ . (3.18) 

7]'^ — 1 rf — rf 

Plugging these expressions into Eq. (I3.14p . we see that the resulting matrices for the 
generators are different from those presented in Eqs. (13. 7p and (13.80 . We must there- 
fore find a similarity transformation that relates both bases. This is facilitated by the 
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observation that a'^' is obtained from B^^^ by conjugation 

al3] = wiB^''^w\ , (3.19) 

where 

w, = B^^\A^^^B^^^yB^^\A^^^B^^^)A^^^ . (3.20) 

Applying this transformation to the other two S4 generators shows that we need a second 
transformation W2 to arrive at the matrices of Eqs. (13. 7p and (13. 8p . However, W2 is 
constrained in that it must not change a^^l, that is W2 a'^^ = a^^' W2- This condition allows 
only matrices of the form 

(X y z\ 
z X y \ . (3-21) 
y z xj 

Demanding W2 to be unitary, we find the solution 

X = ^ (-64 - 15r] - 56r]^ - 46r/^ + 5r/^ - 27r/^) , (3.22) 
y = ^ (73r/ + 1567]^ + 46r/3 + 12r/^ + 1377]^ + llSr^^) , (3.23) 
z = ^ (15 + 34r7 - 35r7^ - 237]^ + Alrj^ - A6r]^) , (3.24) 



with the normalisation factor 
N = [28(-342125 - 349668?] + 283769//^ + 9406?]^ - 501928//^ + 287955//^)] . (3.25) 



The combined similarity transformation W2 Wi leads to the following matrix representation 
for the PSL2{7) triplet 



= w2W,[{A^^^B^^^yB^''\A^^^B^^^)Y wlwl = 0-1 , (3.26) 

\0 -ij 

fO 1 0\ 

fll^l = w2WiB^^^w\wl = 1 , (3.27) 

V oj 

= «;2«;ii?t='l'(A[=*ls[=^l)(A[=*lsMY [(A[='ls[^])25[=^l(A[^]i?[=^l)]' wlwi 

(3.28) 




3] _ (6[3]rf[3]a[3]j,[3])^[3](^[3]^[3]^[3]^[3])-l^t^t ^ _i I ^ -1 1.(3.29) 
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Here we adopt the notation of the "Atlas of Finite Groups" [30] which defines 



bj = ^(-1 + ^v^) , h = ^(-1 - tV7) . (3.30) 

Notice that the sign of the generator fot^l is fixed in the PSL2{7) framework. Hence 
there is only one triplet representation and its conjugate (which is obtained by complex 
conjugation of v^^^). 

Finally, we can perform the basis transformation of Eqs. (13.9113. 12]) to arrive at the 6*4 
generatorcl S, T, U plus the additional generator 

FQi + 1 I 1 I C7 = ^(1 + 3iy/7) , 

V = wv^^^w^ = - 2c7 -2 1 , with <^ ' . ' (3.31) 




C7 



which is necessary to obtain the remaining elements of PSL2{7). With these four (3 x 3) 
matrices, we have found the structure of the embedding ^4 C 5*4 C PSL2{7) in terms of 
the group generators: 

S,T — > A4, 
S*, U — > S4 , 
S, T,U,V — > PSL2{7) = S(168) . 

Furthermore, S, T, U are given in a basis for the triplets that is most suitable for model 
building purposes [29]. 



4 The PSL2{7) sextet in a convenient real basis 



Just like in 5'f/(3), the PSL2{7) sextet x = (Xi? X3? X4? Xs? Xe) generated from the 
symmetric product of two triplets = {ipi:4'2,4'3) and ip' = ("^i; "^2? V's)- With 



Xl = ^l^'l, X2 = V^2V^2, Xs = V'sV^S) 

X4 = 79(^1^2 + V^2^'i) > X5 = ;79(^2V^3 + V^3^2), Xe = 79(^3 V'l +V'lV^3) 



the corresponding PSL2{7) generators for the sextet are 





/ 1 


4 


4 


-2V2 


4^2 


-2V2\ 




4 


1 


4 


-2V2 


-2V2 


4:V2 


1 


4 


4 


1 


4V2 


-2V2 


-2V2 


9 


-2v^ 


-2^2 


4v^ 


5 


2 


2 




4^2 


-2^/2 


-2V2 


2 


5 


2 




{-2^2 


4V2 


-2V2 


2 


2 


5 / 



(4.1) 



(4.2) 



*As for b^^^ in Eq. ([3?28l) . the sign of U is determined such that det{U) = 1. 
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r 



/I o\ 

to 





1 

\0 uj 



u 



(I 














o\ 








1 














1 





























1 














1 





^0 








1 








V 





/ 1 


-4 + C7 


-4 + C7 


2V2C7 


V2(-4 + C7) 


2^2 C7\ 




-4 + C7 


4 


1 


-4^2 C7 


-2^2 


2v^C7 


1 


-4 + C7 


1 


4 


2v^C7 


-2V2 


-4^2 C7 


9 


2V2C7 


-4v^C7 


2^2 07 


2 


-2C7 


5 




v^(-4 + C7) 




-2V2 


-2C7 


5 


-2C7 




2v^C7 


2V2C7 


-4v^C7 


5 


-2C7 


2 / 



(4.4) 



The tilde denotes that this basis for the sextet is complex. Since, unlike in SU{'i), the 
sextet of PSL2{7) is a real representation, it is possible to change to a basis in which all 
generators are explicitly real. In such a real basis, the bilinear invariant 6 ® 6 is trivially 
obtained by adding the squares of all components. In the complex basis, we have to 
find a matrix C such that C x is invariant under all four generators S,T,U,V. A 
straightforward calculation yields 





/2i- V7b7 











iV2b7 


\ 










-■I - V7b7 


-y/Ubr 








1 





-i - V7b7 











-VUbr 


6 





-VUbr 











2ib7 




iV2h 











-2i-2V7bj 







\ 





-^/Tibr 


2ib7 





/ 



Using this matrix together with Eq. (14.11) . we can now easily obtain the invariant combi- 
nation of 3 3 eg) 6. After rearrangement of the terms, we find the PSL2{7) invariant 



6 



(4.5) 



where we have defined the symmetric matrix Xij as 



/{~2i + V7b 



X 



\ 



7)Xl 

V7b7X2 - iV2b7X6 
V7b7X3 - i\/2b7Xi 



j V2 67X5 67X2 - iV2b7XQ a/7 67X3 

{i + Vlb7)x3 + VUb7Xi V2{i + V7b7)x5 
V2{i + V7b7)x5-ib7Xi (^ + v^&7)X2 + 



iV2b7X4 \ 
ib7Xi 



14 67X6/ 
(4.6) 

Assuming that the triplets ip and ip' are Standard Model fermions (for Yukawa terms 
ip' = ip'^ and for Majorana terms ip' = ip) while the sextet x is a flavon field, the Yukawa 
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and Majorana couplings Y^'^'^^Y'^"'^ are generated when the components of x develop 
their VEVs. More intuitively, one can regard the matrix x as a two-index symmetric 
tensor field in the sextet representation which obtains a VEV (x)- 

For practical purposes, especially when constructing and evaluating the sextet flavon 
potential, it is however technically easier to work with the sextet being a six component 
column vector. Furthermore, as the real basis is more convenient, we define the unitary 
transformation 

X = , 



with 



7^ = 





( 


2 


2 




2V2 


-V2 \ 







2iv^ 











(1+0 





—i\/3b7 


i\/2)h-j 







—iy/Qhj 











-V6b7 









-2v^ 


-2^2 


-2\p2 


2 


2 


2 















2^7367 


J 



Then the generators of the sextet take the form 
S = nsn^ = Diag 



Diag(-1, 


-1, 


1,1, 


1,1) 


, 




/ 1 


1 








\/2 0\ 




-1 


-1 








\/2 


1 








-1 







2 










-1 







x/2 ■ 


-x/2 













\o 











2/ 


Diag(l, - 


-1, - 


-1,1, 


1,1) 






/6 











\ 







6 











1 


6 














6 








4 
















3 






Vo 


- 


\/l4 


V2I 


-1 / 



(4.7) 



(4. 



(4.9) 



(4.10) 



(4.11) 



(4.12) 



while the matrix C changes to 



6x6 ■ 



(4.13) 
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5 Towards a realistic model 



As we are working in the real basis for the sextet from now on, we should express x 
the matrix of Eq. (14.61) in terms of Xi see Eq. (14.71) . The result i^ 



X 



6^2 





r 


1 






+ \/367X4 ( 1 


1 
















-1 -1 



4 -i^h 



-1 





1 

(5.1) 



In this form it is evident how the vacuum structure of a sextet flavon field can give rise 
to the Yukawa and Majorana matrices y^^^ ,Y^'^^ . Using the sextet flavons, the (third 
family) effective Yukawa couplings are supposed to originate from terms of the form, 



while the effective Majorana couplings will originate from terms like 



\Yuk„ 



(5.2) 



(5.3) 



where both terms are invariant under PSL2{7), with the indices i,j now labelling the 
entries of the matrix in Eq. (15.11) . In Eqs. (15. 2p and (15.31) we have introduced two different 
sextet flavons, namely xlj^'' xfj"^'' y where the VEV of xli^'' must be aligned along the 



third family direction, (x]^"'^) 



V6i36j3, while XiT""' i^ust develop VEVs which will give 



rise to the TB neutrino mass matrix. 



5.1 Sextet Vacuum Alignment 
5.1.1 The alignment of x^°'^ 

In order to achieve the desired alignment for the VEV of xf/""' ) we require its components 
X^""' ) where a = 1, . . . , 6, to have an alignment of the form 

(Xtb) = - -^Y^^ ■ (0, 0, 0, 04, "5, tte) , (5.4) 

which is left unchanged by the action of the PLS2{7) sextet generators S and U. This is 
clear from Eq. (15. ip where it is observed that the matrices proportional to X4,5,6 all satisfy 
the TB conditions in Eq. (12. 7p . Consequently, the resulting Majorana matrix Y^^^-' is 
invariant under 5* and U : 

o ■ I rpj^ ■ o ~ ^TB 1 ^ ' TB ' ^ ~ ^ TB 

^Note that we are not changing the basis for the triplets. Therefore the matrix x of Eq. (|4.6p is 
identical to the one of Eq. (|5.ip . 
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Since S and U are the symmetries of the tri-bimaximal mass matrix, Yj,^-^ is diagonahsed 
by the tri-bimaximal mixing matrix Utb- We find the following neutrino mass eigenvalues, 

mi = — 67 ^4 + as — iv^ 67 cte , (5.5) 
m2 = 267 0^4 — za/2 67 ae , (5.6) 
= bj + VQ 0(5 + iV^bi . (5.7) 

It is interesting to note that an alignment in which only the sixth component of 
the sextet has a non-vanishing value, leads to a Majorana matrix that is additionally 
symmetric under T. Actually, any PSL2{7) sextet of the form 

x'^l = (0,0,0,0,0,X6) , 

is left invariant under the subgroup S4. Giving a VEV to this field breaks PSL2{7) down 
to 5*4. The complete sextet decomposes as 

6 ^ 1 + 2 + 3' , 

where 3' is the triplet of 5*4 for which det(f/) = — 1. Since in our real basis the matrices 
S, T, U are "quasi" block diagonal, it is easy to check that the directions of the sextet 
that become the doublet and the triplet are 

x'^l = (0, 0, X3, X4, 0, 0) , x'^'' = (Xi, X2, 0, 0, X5, 0) . 

If we want the VEVs of these fields to be unaltered by S and U, we must set (xi) = 
(X2) = (xs) = which brings us back to the structure of {xtb) ^f Eq. (15.41) . 

5.1.2 The alignment of Xi^^^ 

In order to generate a Yukawa matrix which gives mass to only the third generation (top 
quark, bottom quark or tau lepton), we require its components xl"'^? where a = 1, . . . , 6, 
to have an alignment of the form 

(xrof) oc (1 - z) ■ (-1 , - , tV?>lh , , V2 , 0) . (5.8) 

With this vacuum structure for the sextet Xt<^i oiig can easily show that only the (3, 3)- 
entry of the Yukawa matrix in Eq. (15. ip gets filled in. 

5.1.3 The sextet vacuum alignment problem 

To obtain alignments of the type {Xtb) of Eq. ( 15. 4p and (Xtop^) of Eq. ( 15. 8p it is necessary 
to study PSL2{7) symmetric potentials for the fiavon sextet. Even though a detailed 
analysis of the dynamics within our framework goes well beyond the scope of the present 
article, we want to illustrate our procedure which proves successful in finding suitable 
sextet potentials [27]. Following along the lines of [17,19] we consider sextet potentials of 
the form 

6 5 6 

V = -rri^'Y^ xlXi + XI ■ 5Z ^tjkixlXjXlxi ■ (5-9) 

i=l a=0 i,j,k,l=l 
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In general, the quartic part consists of several independent invariants which are denoted 
by a. The coefficients c^^^i are defined by the Clebsch Gordan coefficients of the PSL2{7) 
Kronecker products. As we have two identical sextet fields together with their complex 
conjugates, it is sufficient to consider the square of the symmetric quadratic product 

(6 ® 6), ® (6 ® 6), = (1 + 6 + 6 + 8) (g) (1 + 6 + 6 + 8) , 

which yields a total of six independent invariants, two of which are also SU{3) invariant J§ 
Their explicit derivation is straightforward, though tedious since it requires the knowledge 
of all Clebsch Gordan coefficients for the product (6 ® G)^. Among others, this necessitates 
the construction of the PSL2{7) octet in the S^^\T^^\U^^\V^^^ basis. 

Having calculated the coefficients c^-;,;, the discrete family symmetry does not fix 
the weights Ka with which each invariant enters in the potential. Therefore, from the 
model building point of view, we can choose them so that the desired alignment vectors 
minimise V. To this end, we need to evaluate the first derivatives of each of the six 
invariants for a particular vacuum alignment. Requiring vanishing first derivatives already 
constrains the possibilities for Ka- Next the matrix of second derivatives, i.e. the Hessian, 
must be calculated. Only in the case where the Hessian is positive definite, the potential 
is minimised by the desired alignment. In both cases, i.e. for {xtb) ^^'^ (Xtop)^ can 
find parameter ranges for Kq, which lead to a suitable potential. In hindsight one could 
then argue that a particular messenger sector is responsible for the suppression or absence 
of some of the six independent invariants. 

5.2 Other Yukawa operators 

The complete model should include, in addition to the third family Yukawa couplings, also 
the other operators responsible for the Yukawa couplings of the first and second family 
of quarks and leptons. In order to do this, one may construct the remaining parts of 
the model along the lines of the model in [19] by extending the Yukawa sector to include 
anti-triplet flavons 0, with the leading order operators given by, 

^^'^^ - Axl'^'r.H + ^'^i^.^^r.H + ... , (5.10) 

where the term involving the sextet Xij^^i with the vacuum alignment discussed above, 
leads to the third family Yukawa couplings, while terms involving anti-triplet flavons 
generate the flrst and second family Yukawa couplings. These terms may originate from 
diagrams such as those in Figs. [U^a, 6), leading to a realistic quark and lepton mass 
spectrum. 

similar potential built out of PSL2{7) (anti-)triplet flavons yields quartic invariants which are 
identical to those of SU{3) because the product 3 ® 3 = 1 + 8 has identical Clebsch Gordan coefficients 
in both cases. With solely one flavon field, the only quartic invariant would be the square of the quadratic 
one, (j)'-'' 4>i(j>^'' , so that it is impossible to obtain an alignment for a single (anti-)triplet flavon. 
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6 Conclusion 



We have argued that, in the quest for a candidate discrete family symmetry Gf suitable 
for combining with 5*0 (10) type unification, one should seek groups with the following 
features: (i) Gf should have complex triplet and (real or complex) sextet representations, 
and {ii) d Gf. We have pointed out that the projective special linear group PSL2{7), 
also known as S(168), satisfies both requirements, and furthermore is the unique simple 
finite group containing complex triplets. This makes PSL2{7) ideally suitable as a discrete 
family symmetry for describing quark and lepton masses and mixing in the framework of 
5*0(10) type unified models. 

We have shown how the flavon fields in the sextet representation of PSL2{7) play a 
crucial role both in obtaining tri-bimaximal neutrino mixing as well as in generating the 
third family charged fermion Yukawa couplings. In preparation for physical applications, 
we have derived the triplet representation of PSL2{7) in the basis 5", T, U, V where S, T, U 
are the familiar triplet generators of 5*4 in the diagonal charged lepton basis where T is 
diagonal. We have also derived a convenient real sextet representation, responsible for the 
Majorana couplings of the neutrino operators and the Yukawa couplings of the third family 
charged fermions. We have identified the sextet vacuum alignment which preserves the 
sextet generators S,U and lead to tri-bimaximal neutrino mixing. We have also displayed 
the sextet vacuum alignment which breaks all the generators and gives rise to the large 
third family charged fermion Yukawa couplings. We have briefly described our approach 
to the vacuum alignment problem, which will give rise to both these alignments. 

In conclusion, we have proposed a new and promising class of (S> GUT models based 
on PSL2{7) family symmetry and S'O(IO) type unification, focusing on the symmetry 
aspects of such a model and in particular on the new features arising from the presence 
of flavons in the sextet representation. A complete dynamical model of this kind based 
on PSL2{7) = E(168), including a full analysis of the vacuum alignments of sextet and 
anti-triplet flavons, is in progress and will be presented elsewhere [27]. 
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